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(54) Process for tlie automatic recognition and categorization of defects in pallets or similar 
elements, and corresponding system 


(57) Process and system for the automatic recogni- 
tion and categorization of defects in pallets. The process 
comprises capturing the image of a pallet, digitizing it 
and sending it to a computer, determining the outline of 
this pallet for its subsequent measurement and the pin- 
pointing of the defects, segmenting each one of the el- 
ements of the pallet and detecting and analysing these 
defects in each one of the segmented elements. The 


system is composed of eight standard video cameras, 
a computer with a card for capturing and processing im- 
ages, various sources for illumination, two spotlights 
with intermediate optics, a conveyor belt, two electro- 
mechanical turning-over devices, a protective cabin with 
black paint for the control of the illumination and, finally, 
the set of computer programs developed specifically for 
the automatic analysis and categorization of the pallets. 
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Description 

The present Invention relates to a process for the 
automatic recognition and categorization of defects in 
paliets or similar elements, and to a system which op- 
erates using this process. 

A series of devices and installations for selecting 
and choosing between materials and objects, applicable 
particularly to grading machines and to machines for de- 
tecting defects, is already known. 

Thus, for example, Spanish Patent Application 
503734, in the name of Joan Costa Bofill and Pere Mar- 
tinez Prat, claims a device for selecting and choosing 
between materials and objects for grading machines. 
This device consists of image-receiving cameras which 
are connected to amplifiers, the latter being in turn con- 
nected to analog/digital converters and to synchronizing 
separators. The signal from the converters is sent to a 
processing unit which is linked to the cameras via a 
group of elements which delimit the surface of the object 
to be inspected. 

Spanish Patent Application 8904094, in the name 
of Control Vision Computer, S.A., covers an installation 
for the detection and rejection of defective lids for con- 
tainers which consists of a conveyor track by means of 
which the lids are transferred and in which at least one 
width-wise interruption has been made, a source of light 
and a video camera which face each other and are ar- 
ranged one on either side of the strip, the video camera 
sending its signal to an artificial-vision system with 
means for detecting and locating the existence of a hole 
in a lid, ordering expulsion means to remove the lid with 
the hole from the conveyor track. This installation is ap- 
plicable to containers for perishables. 

Spanish Utility Model 9302777, in the name of Apli- 
caciones Ferricas, S.A., covers an artificial-vision ma- 
chine for quality control in the manufacture of ceramic 
paving and flooring components. This machine includes 
a belt for conveying the components to be analysed 
which, by means of a positioning device, halts them, 
centres them and locates them with respect to a con- 
stant and uniform Illumination apparatus which, via a vi- 
sion apparatus, receives the variables of the component 
to be analysed and checks them against the corre- 
sponding variables of an electronic card of a computer 
which actuates actuators of the system for categorizing 
the components. 

European Patent Application 0,460,431, in the 
name of Rautaruukii OY, relates to a method for the op- 
tical inspection of a band in which the aim of the method 
and the corresponding equipment is to detect defects in 
the surface of said band. 

Furthermore, in 1994 the Applicant developed an 
initial prototype intended for the automatic categoriza- 
tion of a CHEP 800 X 1 200 pallet by means of artificial- 
vision techniques. 

The process and the system for the automatic rec- 
ognition and categorization of defects in pallets or sim- 


ilar elements, which Is the subject of this invention, has 
two general objectives, namely, firstly, to provide the 
technical bases for developing a complete system for 
automatic categorization of pallets which is capable of 
5 automatically recognizing all the defects which said pal- 
let may contain. Secondly, it is an aim of the present 
invention to develop a system capable of automatically 
categorizing a series of defects in the said pallets, such 
as: 

10 

the absence of a component; 

a broken or split component; 
a component which has been displaced from its po- 
sition by more than the specified amount; 
?5 - a component which has splintered beyond set tol- 
erances; 

a component with a significant loss of volume; 
a cracked component or one with fissures outside 
set tolerances; 
20 - component dimensions outside set tolerances; 

defects in the wood: knots, stripped-bark areas, 
slits, etc.; 

defects in the nails. 
25 Of these, three are of particular interest, namely: 

1 . A transverse break in the upper platform; 

2. The detection of longitudinal fissures greater than 

300 mm; and 

30 3. The loss of more than one third of the volume in 
the central stud. 

In order to achieve the said general objective, the 
authors of the invention considered the following five 

35 phases: 

1. The development of an appropriate image-cap- 
turing system; 

2. The setting-up and testing of extraction algo- 
40 rithms for edges and segmentation; 

3. The design of a device for recognizing defects; 

4. The development of a user environment; and 

5. The validation of the system by means of the 
studying of a sample of defective pallets. 

45 

The subject of the present invention is therefore a 
process for the automatic recognition and categoriza- 
tion of defects in pallets which is characterized in that it 
comprises the stages of: 

50 

• indirectly illuminating the pallet against a dark back- 
ground, capturing its image, digitizing it and sending 
it to a computer; 

• determining the outline of the pallet for its subse- 
55 quent measurement and the pinpointing of the de- 
fects; 

segmenting each one of the elements of the pallet, 
individualizing these elements of the image; 
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detecting and analysing the defect or defects found 
in each one ol the segmented elements, taking into 
account the error specifications which have been 
catalogued. 

According to the invention, this process is carried 
out on all the possible defects of each one of the ele- 
ments of the pallet, a specific analysis subsequently be- 
ing carried out on each defect detected in order to de- 
termine whether this is a catalogued defect. 

In accordance with the invention, the analysis of the 
results obtained is based on a set of digitized images of 
the pallets, these images being stored in computer files; 
the detection and the analysis of the defects is carried 
out with dynamic error specifications to which parame- 
ters can be applied, with the aim of obtaining various 
quality criteria. 

Lastly, in accordance with the invention, the proc- 
ess is characterized in that the determination of the out- 
line of the pallet involves an operation of highlighting and 
enhancing the original image of this pallet and an auto- 
matic thresholding of the histogram of the highlighted 
image. 

A further subject of the present invention is a sys- 
tem, based on artificial-vision techniques, for carrying 
out the above-described process, which system is char- 
acterized in that It consists of an automatic belt of trans- 
port chains (3) with an illumination system and a non- 
interlinked set of video cameras (5) with their respective 
optics, the cameras being located at a predetermined 
mutual distance, with devices (6) for turning the pallets 
over, and in that It includes a card for digitizing a video 
signal. 

According to the invention, the computer specifical- 
ly analyses the defects found, verifying whether a cata- 
logued defect is involved, revealing, by means of an 
identifier, the defect found and indicating the condition 
of the element of the pallet examined. In addition, ac- 
cording to the Invention, the computer includes a series 
of files with the digitized images of the pallet which is 
the subject of the examination. 

The image-capturing system used for the viability 
study consists of: 

• a non-interlinked set of video cameras with their re- 
spective optics; 

• a Pentium PCI PC -type computer; 

• a card for capturing and processing an image with 
a resolution of 768 x 576 pixels and 258 levels of 
luminous intensity on the white/black scale; 

• a package of programs for processing digital imag- 
es and artificial vision for the automatic categoriza- 
tion and measurement of the pallets. 

The illumination used in the image-capturing sys- 
tem was of utmost importance, satisfactory results being 
obtained with light not incident directly on the pallet and 
the use of a black-background screen which enhances 


contrast. 

The cameras used are of the standard video-signal 
type, supplemented with C mount optics. 

The use of a powerful computer, which is necessary 
5 for this type of processing, together with optimization of 
the processing algorithms made it possible to achieve 
an acceptable response time. 

The capture processing consists in the computer 
suitably illuminating the pallet, taking the image, digitiz- 
^0 ing it and turning the pallet over so that the various 
processing operations can be carried out. It is possible 
to store the images of the defective pallets for subse- 
quent statistical analysis of defects and study of the 
causes thereof. 

The next step is to find the outline of the pallet in 
order to be able to identify it, obtain its measurements 
and pinpoint the defects. This is a critical step in obtain- 
ing a satisfactory result in the next stage of identifying 
the defects. In addition, this processing has to be very 
20 rapid, and therefore a compromise was sought between 
the quality of the edges obtained and the calculation 
time employed. 

After a number of tests using various methods of 
obtaining edges, the process consists in: 

25 

a. an operation of highlighting and enhancing the 
contrast of the original image of the pallet; 

b. the automatic thresholding of the histogram of the 
image highlighted. 

30 

A description will now be given of the subject of this 
invention, with the aid of the drawings, although it must 
be understood that these represent no more than a pre- 
ferred embodiment of the invention which must in no 
55 way be regarded as limiting its scope. 

Figure 1 A shows a highlighted original image of a 
pallet which Is the subject of an examination; 
Figure 1 B shows the extraction of the outline of the 
40 pallet in Figure 1 A; 

Figure 2A shows a highlighted original image of a 
second pallet; 

Figure 2B shows the extraction of the outline of the 
pallet in Figure 2A; and 

45 - Figure 3 shows the image of the pallet picked up in 
Figure 1A, which reveals the segmentation of the 
elements, the maxima of the vertical graph showing 
the beginning and end of each plank; and 
Figure 4 shows a view in lateral section of the sys- 

50 tern which is the subject of the invention. 

With reference to the figures, once the outline of the 
pallet (1) has been obtained, the next step is the seg- 
mentation of each plank (2), since the processing for 
55 identifying the defects Is carried out individually for each 
element making up the pallet. Segmentation consists 
basically in individualizing the elements of the image, 
for example distinguishing each one of the five upper 
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pianks (2) of the pallet. In this processing, the algorithm 
used offers excellent results, virtually 100% of the sam- 
ples used being segmented and the response time be- 
ing very rapid. 

After the segmentation of each element, the next 
step Is the detection and analysis of defects. This 
processing consists in checking the surface of each 
plank (2), looking for possible defects. Once a possible 
defect has been detected, it is compared with a knowl- 
edge base which confirms whether or not the candidate 
is a critical defect. At this point, account is taken of the 
error specifications given by the user, which are dynam- 
ic and may have parameters applied to them, i.e. it is 
possible to modify the precision employed in recogniz- 
ing defects, this being an essential aspect if we wish to 
have various quality criteria according to client require- 
ments. 

With reference to Figure 4, this shows a lateral view 
of the system for the automatic recognition and catego- 
rization of defects in pallets, and it must be understood 
that this is a preferred embodiment of the invention but 
in no way limits it. The system comprises an automatic 
belt of transport chains (3) which passes through a 
chamber (4) provided with an illumination system with 
a set of eight non-interlinked video cameras (5) with their 
respective optics and a predetermined distance be- 
tween them. Two turning-over devices (6) are also pro- 
vided on the conveyor belt, their function being to turn 
the pallet (1 ) over so that it is possible to inspect its other 
side. Once the pallet has been completely inspected, 
the second turning-over device (6) rotates the pallet (1 ) 
back to its normal position. 

Specifically, the system which is the subject of the 
invention will carry out the following automatic measure- 
ments of the pallets or similar elements: 

the eight external dimensions of the pallet, of its up- 
per or lower part; 

- the width of the spaces, which will be measured 
along the entire pallet. Therefore, two measure- 
ments are carried out for each gap between studs, 
making it possible to obtain six measurements per 
space and twelve measurements in total; 

- the height of the said spaces, which height will be 
measured on the inner ridges of the six external 
studs, therefore eight measurements in total; and 
the height of the apertures which will be measured 
on the inner ridges of the studs corresponding to 
the aperture, therefore eight measurements in total. 
Ultimately, the system will have to provide thirty 
measurements for each pallet. It should be pointed 
out that the maximum error in the measurements 
will be set, in the worst of cases, at between ±5 mm 
with respect to the actual value. 

It is very important to stress the process which has 
been used for categorizing the defects and which, as we 
have seen, consists of two stages or steps: (a) detection 


of the defects, and (b) categorization of the defects. 

The first stage is responsible for detecting all the 
possible defects which may appear in an individual ele- 
ment of the pallet under study. It is vital that, at this stage, 

s no type of defect should "escape". Next, each defect de- 
tected is specifically analysed to verify whether it is or 
is not a defect which has been catalogued. This specific 
analysis is determined, as indicated previously, by the 
user's specifications, i.e. these are defect specifications 

10 which may be defined externally, according to the types 
of defects it is desired to recognize. 

In short, with the first stage, all the possible defects 
of a pallet are detected and, with the second stage, only 
those defects which are actually required are recog- 

'5 nized or categorized. 

Lastly, the system reveals the defects found by 
means of a "red" identifier which Is located just above 
the defect and a message which indicates whether the 
element in question is or is not in a satisfactory condi- 

20 tion. 

The analysis of results obtained is based on a set 
of digitized images of pallets stored in files. The proto- 
type categorizes defects appearing in the following ele- 
ments of the pallet: 

25 

I. Planks (external and Intermediate) on the surface; 

II. Studs; 

III. Runners. 

30 The system makes it possible directly to read the 
files with the stored pallet images and directly to capture 
in real time any pallet to be categorized. 

The results obtained with the sample set of pallets 
are shown In the following table: 

35 


TABLE 


ANALYSIS OF RESULTS 


Inspection Time 

Success Rate 

Surface planks 

2.9 sec 

100% 

Central stud 

1.2 sec 

100% 

Runners 

1 .7 sec 

100% 


The above results deserve some commentary: the 
success percentage was calculated on the number of 
defects detected. Overall, in all the sample pallets used, 
all the existing defects are detected, including defects 
such as knots in the wood, holes and blemishes. Since 
the set of defects is limited to the three types of defects 
already indicated, in the second stage the system (by 
virtue of the focused approach introduced, which spec- 
ifies the final defects to be categorized) categorizes only 
those defects, amongst all the defects previously detect- 
ed, which correspond to the three types of defects al- 
ready indicated. 

The times achieved for categorization are essential- 
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ly low, by virtue of highly refined debugging of the cate- 
gorization programs. Thus, the above table shows that 
the maxinnum categorization time for each pallet image 
or shot does not exceed 3 seconds. 

Naturally, the final time for categorizing a pallet will s 
depend on the number of images or shots to be taken 
for each pallet. In a first approximation, it was estimated 
that around 8 shots per pallet would be required: front, 
rear, two sides and several additional shots from various 
angles. Since these shots may be taken independently io 
of one another, the parallel running of the processing for 
categorizing each shot or image makes it possible for 
the total time for categorizing a pallet to be equal to the 
time for categorizing the slowest shot. 

In addition, the use of a successive refining struc- 
ture reduces the average pallet-categorization time. 
This idea consists basically in carrying out the entire de- 
fect-identification processing only on those pallets 
which are defective. It is not necessary to carry out the 
entire processing on those which are in satisfactory con- 20 
dition. To this end. an initial 'approximate" identification 
indicates whether the pallet which is the subject of the 
identification is a candidate which will have some defect 
or other If this js the case, the next step is a "finer" iden- 
tification which makes it possible to detect the possible 2S 
defect. If this is not the case, the pallet is identified as 
being "in a satisfactory condition" and the process 
moves on to the next pallet. In this way, not all the pallets 
are subjected to the complete processing, only those 
which are defective. 30 

Furthermore, if the defect-detection requirements 
are relaxed, which may be necessary so that the number 
of defective pallets needing repair does not increase ex- 
cessively, i.e. if less strict defect specifications are being 
employed, the categorization time would also drop in 35 
proportion. It is therefore appropriate to emphasize the 
importance of it being possible to apply parameters to 
the system as far as the specifications for the defects to 
be detected are concerned. 

This application of parameters makes it possible 40 
easily to adapt the system to all types of pallets, manu- 
ally or automatically choosing the set of pallets to be 
examined. The application of parameters also makes it 
possible to define various quality standards according 
to the criteria required by each client, as a function of 45 
the final quality of the pallet to be received by each cli- 
ent. This affects inspection-time performance since, if a 
quality standard is established which is stricter than an- 
other, the recognition time will increase in proportion. 

so 

ADVANTAGES OF USING AN AUTOMATIC 
RECOGNITION SYSTEM 

One of the objectives it is aimed to achieve with the 
installation of an automatic categorization system is un- ss 
doubtedly the increase in product quality. There are sev- 
eral reasons which support the attainment of this objec- 
tive: 
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• The categorization criteria are obiectives: the man- 
ual recognition processing leaves the categoriza- 
tion decision to the judgement of the operator in- 
specting the pallet. This judgement may vary be- 
tween two operators or even in one and the same 
operator, depending on various factors such as the 
lack of lighting, fatigue, concentration, etc. An au- 
tomatic categorization system always has the same 
criteria. 

• Parameters can be applied to the categorization cri- 
teria : this feature makes It possible to establish var- 
ious quality standards, according to client require- 
ments, thereby optimizing not only the categoriza- 
tion time but also the quality of the pallets received 
by clients. Some of the variables to which parame- 
ters may be applied are: a) the measurements of 
the various models, b) the criteria for categorizing 
defects, using the knowledge base containing infor- 
mation on various quality standards. 

• Continuous operation : once installed, these sys- 
tems are able to operate 24 hours a day, since ex- 
ternal conditions do not change. The number of pal- 
lets processed per day would therefore increase 
considerably 

• Statistical analysis and monitoring of the pallets : au- 
tomatic recognition of defects makes It possible to 
control the pallet cycle. Once the pallet has been 
categorized, it is possible to add a bar code speci- 
fying the type of defect it contains and its location. 
Therefore, on arrival at the repair shop, it is unnec- 
essary to inspect it again - it is simply a matter of 
reading the code and then repairing the defect. 
Moreover, the code could contain a record of the 
date and place of the most recent inspection, which 
could therefore make it possible to control the life 
cycle of the pallets, clients who damage the material 
most, consignments containing a characteristic de- 
fect, etc. 

• Quality control immediately after manufacture : the 
system is implicitly measuring the dimensions of 
each pallet. Inspection of the measurements Is an 
even more rapid processing operation than the de- 
tection of defects. This can be used directly after 
the pallets come off the assembly line and those 
which do not have the correct measurements (with 
the required approximation) may be rejected. In this 
way it is possible to certify the quality of the pallet 
in terms of measurements and absence of defects. 

CONCLUSIONS 

I. An analysis was made of the viability conditions 
for a subsequent project for automatic categoriza- 
tion of a series of defects, the conclusion arrived at 
being that said global categorization is possible with 
the algorithms developed in this viability study 

II. A system has been constructed which is capable 
of recognizing, in real time, the subset of defects 
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selected from the total set of defects, this subset 
having been used to evaluate the system's perform- 
ance with a set of defective pallets. The average 
recognition time is under 3 seconds. However, ac- 
count must be taken of the fact that several images s 
have to be taken for each pallet. This does not mean 
that the time for inspecting the complete pallet will 
increase, since the parallel running of algorithms 
makes it possible to opt for a multiprocessor archi- 
tecture. 

III. The application of parameters stands out as one 

of the principal advantages of the system, it being 
possible to move on rapidly from one model to an- 
other or to specify another quality standard. 

Claims 

1. Process for the automatic recognition and catego- 
rization of defects in pallets or similar elements, 
characterized in that it comprises the stages of: 

indirectly illuminating the pallet against a dark 
background, capturing its image, digitizing it 
and sending it to a computer; 
- determining the outline of the pallet for its sub- 
sequent measurement and the pinpointing of 
the defects; 

segmenting each one of the elements of the 
pallet, individualizing these elements of the im- 
age; 

detecting and analysing the defect or defects 
found in each one of the segmented elements, 
taking into account the error specifications 
which have been catalogued. 

2. Process according to Claim 1 . characterized in that 
detection Is carried out on all the possible defects 
of each one of the elements of the pallet, a specific 
analysis subsequently being carried out on each 40 
defect detected in order to determine whether this 

is a catalogued defect. 

3. Process according to Claims 1 and 2, characterized 

In that the analysis of the results obtained is based ^ 
on a set of digitized images of the pallets, these im- 
ages being stored in computer files. 

4. Process according to Claim 1 , characterized in that 
the detection and the analysis of the defects is car- so 
ried out with dynamic error specifications to which 
parameters can be applied, with the aim of obtain- 
ing various quality criteria. 

5. Process according to Claim 1 , characterized In that ss 
the determination of the outline of the pallet involves 

an operation of highlighting and enhancing the orig- 
inal image of the pallet and an automatic threshold- 


ing of the histogram of the highlighted image. 

6. System for the automatic recognition and categori- 
zation of defects in pallets or similar elements, 
which operates using the process according to 
Claims 1 to 5, characterized in that it consists of an 
automatic belt of transport chains (3) with an illumi- 
nation system and a non-interlinked set of video 
cameras (5) with their respective optics, the camer- 
as being located at a predetermined mutual dis- 
tance, with devices (6) for turning the pallets over, 
and in that it includes a card for digitizing a video 
signal. 


so 


10 


'5 7. System according to Claim 6. characterized in that 
the computer specifically analyses the defects 
found, verifying whether a catalogued defect is in- 
volved, revealing by means of an identifier the de- 
fect found and indicating the condition of the ele- 
20 ment of the pallet. 

8. System according to Claim 6. characterized in that 
the computer includes a series of files with the dig- 
itized images of the pallets. 

25 

9. System according to Claims 6 to 8, characterized in 
that it is possible to apply parameters thereto. 


30 
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